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Abstract 

• We survey research that studies the connection between the computational complexity 

I of optimization problems on the one hand, and the duality gap between the primal and 

• ■ dual optimization problems on the other. To our knowledge, this is the first survey that 

"•pi I connects the two very important areas. We further look at a similar phenomenon in finite 

■ model theory relating to complexity and optimization. 
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^ : 1 Introduction 

In optimization problems, the duality gap is the difference between the optimal solution 
values of the primal problem and the dual problem. The relationship between the duality gap 
^ , and the computational complexity of optimization problems has been implicitly studied for 

' the last few decades. The connection between the two phenomena has been subtly acknowl- 

edged. The gap has been exploited to design good approximation algorithms for NP-hard 
optimization problems [TCj I24j . However, we have been unable to locate a single piece of 
literature that addresses this issue explicitly. 



This report is an attempt to bring a great deal of evidence together and specifically address 
this issue. Does the existence of polynomial time algorithms for the primal and the dual 
problems mean that the duality gap is zero? Conversely, does the existence of a duality gap 
imply that either the primal problem or the dual problem is (or both are) NP-hard? Is there 
an inherent connection between computational complexity and strong duality (that is, zero 
duality gap)? 

Vecten and Fasbender (independently) were the first to discover the optimization duality [13j . 
They observed this phenomenon in the Fermat-Torricelli problem: Given a triangle Ti, find 



the equilateral triangle circumscribed outside Ti with the maximum height. They showed 
that this maximum height H is equal to the minimum sum of the distances from the vertices 
to Ti to the Torricelli poin^. Thus, this problem enjoys strong duality. 

The apparent connection between the duality gap and computational complexity was con- 
sidered more than thirty years ago. Linear Programming (LP) is a well known optimization 
problem. In the mid 1970s, before Khachiyan published his ellipsoid algorithm, LP was 
thought to be polynomially solvable, precisely because it obeys strong duality; that is, the 
duality gap is zero. For a good description of the ellipsoid algorithm, the reader is referred to 
the good book by Fang and Puthenpura [9]. Strong duality also places the decision version 
of Linear Programming in the class NP n CoNP; see Lemma [9] below. 

We should stress that this is not a survey on Lagrangian duality or any other form of opti- 
mization duality. Rather, this is a survey on the connections and relationships between 
the computational complexity of optimization problems and duality. 



2 Definitions 

A few definitions are provided in this section. 

Definition 1. (1Di(r).- Decision problem corresponding to a given minimization problem) 

Given. An objective function /(x), as well as mi number of constraints g(x) = b and m2 
number of constraints h(x) > c, where x G M" is a vector of variables, b G M™^ and c G R™'^ 
are constants. Also given is a parameter r G M. 

To Do. Determine if the set T = %, where F = {x. : g(x) = b, h(x) > c and /(x) < r}. 

(Analogously, if the given problem is maximization, then 

= {x : g(x) = b, h(x) > c and /(x) > r}.) ■ 

T is the set of feasible solutions to the decision problem. 

Remark 2. A word of caution: For decision problems, the term "feasibility" includes the 
constraint on the objective function; if the objective function constraint is violated, the problem 
becomes infeasible. 

Definition 3. (Lagrangian Dual) 

Suppose we are given a minimization problem Pi such as 

Minimize /(x) : X ^ R (X C R"), 

subject to g(x) = b, h(x) > c, (1) 
where x G R", b G R"'^ and c G R'"^ . 

Let u G R™^ and v G R™^ be two vectors of variables with v > 0. Let e = (g, h). Assume 
that b = [6i 62 ■ ■ ■ bm-i^ o-nd c = [ci C2 ■ ■ ■ £^2]^ '^'"^ column vectors. The feasible region for 
the primal problem is X . 

^The Torricelli point X is indeed the one with the least sum of the distances \AX\ + \BX\ + \CX\ from the 
vertices A, B and C of Ti. 
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For a given primal problem as in Pi, the Lagrangian dual problem P2 is defined as follows: 

Maximize 9{u,v) 
subject to V > 0, where 

mi m2 (^) 

^(u, v) = inf {/(x) + M9i{^) -h) + Y, W - Ci)}- 

i=i j=i 

■ 

Note that gi{x) — hi = [hj{x) — cj > 0] is the z*^ equahty [j^^ inequahty] constraint. 



3 Background: Duality and the classes NP and CoNP 

( Note to Reviewers : Hope the definitions and explanation in this section is sufficient. If not, 
please let us know where we should expand and elaborate.) 

We now turn our attention to the relationship between the duality of an optimization problem, 
and membership in the complexity classes NP and CoNP of the corresponding set of decision 
problems. Decision problems are those with yes/no answers, as opposed to optimization 
problems that return an optimal solution (if a feasible solution exists). 

Corresponding to Pi defined above in ([T]), there is a set Di of decision problems, defined as 
Di = {Di{r) I r G M}. The definition of Di[r) was provided in Def. [TJ 

Let us now define the computational classes NP, CoNP and P. For more details, the interested 
reader is referred to either [2] or [21]. We begin with the following well known definition: 

Definition 4. NP (respectively P) is the class of decision problems for which there exist non- 
deterministic (respectively deterministic) Turing machines which provide Yes/No answers in 
time that is polynomial in the size of the input instance. In particular, for problems in P 
and NP, if the answer is yes, the Turing machine (TM) is able to provide an "evidence" (in 
technical terms called a certificate ), such as a feasible solution to a given instance. 

The class CoNP of decision problems is similar to NP, except for one key difference: the 
TM is able to provide a certificate only for no answers. 

From the above, it follows that for an instance of a problem in NP fl CoNP, the corresponding 
Turing machine can provide a certificate for both yes and no instances. 

For example, if Di{r) in Def. [1] above is in NP, the certificate will be a feasible solution; that 
is, an X G which obeys the constraints 

g(x) = b, h(x) < c and /(x) > r. (3) 

On the other hand, if Di{r) G CoNP, the certificate will be an x G M" that violates at least 
one of the rui + m2 + 1 constraints in ([3]). 

Remark 5. For problems in NP, for Yes instances, extracting a solution from the certificate 
is not always an efficient (polynomial time) task. Similarly, in the case of CoNP, pinpointing 
a violation from a Turing machine certificate H is not guaranteed to be efficient either. 

^We thank WenXun Xing and PingKe Li of Tsingiiua University (Beijing) for ponting out the above. 
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Remark 6. P C NP, because any computation that can he carried out by a deterministic 
TM can also be carried out by a non- deterministic TM. The problems in P are decidable 
deterministically in polynomial time. 

The class P is the same as its complement Co-P. That is, P is closed under complementation. 

Furthermore, Co-P (= P) is a subset of CoNP. We know that P is a subset of NP. Hence 
P C NP n CoNP. Thus for an instance of a problem in P, the corresponding Turing machine 
can provide a certificate for both yes and no instances. 

We are now ready to define what is meant by a tight dual, and how it relates to the intersection 
class of problems, NP n CoNP. Note that for two problems to be tight duals, it is sufficient 
if they are tight with respect to just one type of duality (such as Lagrangian duality, for 
example) . 

Definition 7. Tight duals and the class TD. Two optimization problems Pa and Ph are dual 
to each other if the dual of one problem is the other. 

Suppose Pa and Pi, are dual to each other, with zero duality gap; then we say that Pa and Pi, 
are tight duals. For any r € M, let Da{r) and Di,{r) he the decision versions of Pa and Pb 
respectively. 

Let TD be the class of all decision problems whose optimization versions have tight duals. 
That is, TD is the set of all problems Da{r) and Di,{r) for any r € R. 

Remark 8. A word of caution. Tight duality is not the same as strong duality. For strong 
duality, it is sufficient if there exist feasible solutions to the primal and the dual such that the 
duality gap is zero. For tight duality to hold, we also require that the primal problem Pa and 
the dual problem Pi, be dual to one another. 

One way in which duality gaps are related to the classes NP and CoNP is as follows: 
Lemma 9. fM/ TD C NP n CoNP. 

Prom Remark[6]and Lemma[9l we know that both TD and P are subsets of NP n CoNP. But 
is there a containment relationship between TD and P? That is, is either TD C P or P C 
TD? This is the subject of further study in this paper, with particular reference to Lagrangian 
duality. 

Remark 10. We should mention that in several cases, given a primal problem P, even if we 
are able to find a dual problem D such that the dual of P is D, it does not necessarily follow 
that the dual of D is P. That is, the dual of the dual need not he the primal. P and D are 
not necessarily duals of each other. We do not include such {P, D) pairs in TD. Among 
primal-dual pairs of problems, TD is a restricted class. 

4 Lack of Strong Duality results in NP hardness 

In this section, we will review results from the literature, which show that the lack of strong 
duality imply that the optimization problem in question is NP-hard, assuming that the primal 
problem obeys the constraint qualification assumption as stated below in Def. O Here we 
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work with Lagrangian duality. Results for other types of duality such as Fenchel, geometric 
and canonical dualities require further investigation. 

Let us define what we mean by weak duality (as opposed to tight duality and strong duality) : 

Definition 11. Given a primal problem Pi and a dual problem P2, as defined in Def. 0, the 
pair (Pi,P2) is said to obey weak duality if 9{u,v) < /(x), for every feasible solution x to 
the primal and every feasible solution (u, v) to the dual. ■ 

Definition 12. In Def. {Tip, if 

(i) the inequality is replaced by an equality, and 

(a) there exist a primal feasible solution 5i and a dual feasible solution (u, v) such that the 
equality in (i) holds, 

then the pair (Pi,P2) is said to be obey strong duality. 

The following theorem from [3] guarantees that the feasible solutions to Lagrangian dual 
problems ([1]) and ([2]) indeed obey weak duality: 

Theorem 13. If x is a feasible solution to the primal problem in ([ip and (u, v) is a feasible 
solution to the dual problem in then /(x) > 9{u,v). ■ 

We shall now define a special type of convex program, called a convex program with constraint 
qualification, which is one with an assumption about the existence of a feasible solution in 
the interior of the domain. 

Definition 14. Convex program (convex optimization problem). 

Given. A convex set X C M", two convex functions /(x) : ^> M and g(x) : M" M'"^ 
as well as an affine function h(x) : M" —?■ . 

To do. Minimize /(x), subject to g(x) < 0, h(x) = and x € X. 
Definition 15. Convex program with constraint qualification. 

Same as the optimization problem in Def. \14l except that we include the following constraint 
qualification assumption. There is an xq ^ X such that g(xo) < 0, h(xo) = 0, and € 



(Note: Of course, the functions above can be written in the same form as in Def. [T]and[2j In 
such a case, we can define (g(x) — b) to be a convex function and (h(x) — c) to be an affine 
function, where b € M^^ and c G M'"^ ) 

Definition 16. In Def [I^ if any of the (m + l) functions /(x) : M" ^ R and g(x) : M" 
M"^^ is not convex, then the optimization problem is said to non-convex. 

For the remainder of this section, we will assume primal constraint qualification; that is, 
we assume that constraint qualification is applied to the primal optimization problem. The 
following theorem provides sufficient conditions under which strong duality can occur: 

Theorem 17. Strong Duality If (i) the primal problem is given as in Def. \14\ and 

(a) the primal and dual problems have feasible solutions, then the primal and dual optimal 
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solution values are equal (that is, the duality gap is zero): 



mf{/(x) : X e X, g(x) < 0, h(x) = 0} = sup{6l(u, v) : v > 0}, 

mi m2 / .N 

9{u,v) = inf {/(x) + Vn,5,(x) + V t;j/ij(x)}, 

i=i j=i 

where 0{u, v) is the dual objective function. 

Using the contrapositive statement of Theorem [T71 we get the following result: 

Corollary 18. (to Theorem \17\) If there exists a duality gap using Lagrangian duals, then 
either the primal or the dual is not a convex optimization problem. (Remember, we are 
assuming constraint qualification.) 

The Subset Sum problem is defined as follows: Given a set S of positive integers {di , ^2 , • • • , dfc} 
and another positive integer do, is there a subset P of S, such that the sum of the integers in 
P equals do? 

Using a polynomial time reduction from the Subset Sum problem to a non-convex optimization 
problem (see Def. [T6]) . Murty and Kabadi (1987) showed the following: 

Theorem 19. 119] If an optimization problem is non-convex, it is NP-hard. 

In certain cases, non-convex problems have an equivalent convex formulation, for example, 
through strong duality. Such a dual transformation, where a convex problem i? is a dual 
of a non-convex problem A such that the duality gap between them is zero, is called hidden 
convexity In such cases, the reformulated convex problem is also NP-hard; otherwise 

the primal non-convex problem can be solved efficiently, thus violating Theorem 1191 

Definition 20. Standard Quadratic Program (SQP) fdj/. Minimize the function x^Qx, where 
x G A (the standard simplex in M" j. The n vertices of A are at a unit distance (in the 
positive direction) along each of the n axes ofW^. Q is a given symmetric matrix in M"^". 

The converse of Theoremll9lis not true. A convex optimization problem in general is NP-hard, 
SQP being an example. SQP is non-convex; however in [U], Bomze and de Clerk prove that 
it has an exact convex reformulation as a copositive programming problem. SQP is known to 
be NP-hard, since its decision version contains the max-clique problem in graphs as a special 
case. From this, it follows that copositive programming is also NP-hard; see [7] for more on 
this topic. 

From Corollarv 1181 and Theorem 1191 it follows that 

Theorem 21. Assuming constraint qualification, if there exists a duality gap using La- 
grangian duals, then either the primal or the dual is NP-hard. 

These results are true for Lagrangian duality. For other types of duality such as Fenchel, 
geometric and canonical dualities, this requires further investigation. 
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5 Does Strong Duality Imply Polynomial Time Solvability? 



At this time, such a proof (of whether a duahty gap of zero imphes polynomial time solvability 
of the primal and the dual problems) appears possible only for very simple problems, since 
estimating the duality gap appears extremely challenging for many problems. 

Some of the problems where this is true include Convex Programming (and in particular, 
Linear Programming), where both the primal and the dual problems are convex; the polyno- 
mial time algorithms are derived from interior point methods. For example, see the book by 
Nesterov and Nemirovskii [20j. 

In a recent paper P2], we have demonstrated an additional computational benefit arising 
from strong duality. Primal-dual problem pairs that are polynomially solvable and obey 
strong duality can be solved by a single call to a decision Turing machine, that is, a Turing 
machine that provides a Yes/No answer (if the answer is yes, then it can provide a feasible 
solution which supports the Yes answer). Previously, it was only known that optimization 
problems require multiple calls to a decision Turing machine (for example, doing a binary 
search on the solution value to obtain an optimal solution). For more details, the reader is 
referred to [17J. 

More investigation is needed to answer the question Does Strong Duality Imply Polynomial 
Time Solvability? in a general setting. 

However, there have been some results recently using Canonical duality for certain types of 
quadratic programs [lOl |25] . Consider a standard quadratic programming (primal) problem: 

Minimize f (x) = —x^Ax + b^x 
subject to b^x < Cj, 1 < i < m, 

where A = G M"^", bj G M" for 1 < i < m, and x G M". ^ is a symmetric matrix. We 
can write the Lagrangian function as 



L(x, A) = ^^^Ax + bo + Xihi x - ^ 

V i=l / i=l 



Kci, Xi > 0. (6) 



The first order necessary condition among the Karush-Kuhn- Tucker (KKT) conditions yields 

m 

Ax -|- bo + Ajbj = 0, from which we get the value of x as 



1=1 



X = -A-^ bo + A^b, , (7) 



i=l 



assuming that A is an invertible matrix. Substituting this value of x back into the Lagrangian 
function yields 

^ / m \^ / m \ m 

Q(A) = -- bo + J^Aibi A^Mbo + ^Aibi -^A,Ci. (8) 

\ t=i / \ j=i / i=i 
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We then get a dual problem, the canonical dual P*^, as follows: 



Maximize Q(X), 

(9) 

subject to A y and A > 0. 



(The relation ^ ;^ means the matrix A should be positive definite.) 

The positive definiteness of A has been found to be a sufficient condition. Whether it is a 
necessary condition is not known yet. 

Now, Q{X) is a concave function, to be maximized in the dual variable A; hence P'^ can be 
solved efficiently, since we are maximizing a concave function. Thus in general, we can get a 
lower bound for the primal problem quickly; and in some cases, we can get a strong dual with 
zero duality gap, which provides an optimal solution for the primal problem in polynomial 
time. 

Canonical duality was first developed to address the problem of (possibly large) duality gaps 
in Lagrangian duality in the context of problems in analytical mechanics [11]. Furthermore, 
large duality gaps can also be found while solving non-convex optimization problems using 
Fenchel-Morrow-Rockafellar duality. The so-called canonical dual transformation can be used 
to formulate a strong-dual problem (that is, one with a zero duality gap). Quoting from [llj, 
the primal problem can be made equal to its canonical dual in the sense that they have the 
same KKT points. 



5.1 Preliminary Results: Canonical Duality and the Complexity Classes 
NP and CoNP 

We consider quadratic programming H problems with a single quadratic constraint (QCQP) 
[25]. The primal problem Pq is given by: 

Minimize Pfx) = — x^j4x — f-^x 

1 , ' m 

subject to —X Px < /i, 

where A and B are non-zero n x n symmetric matrices, f € R", and € R. {A, B, f and 
are given.) Corresponding to the primal Pq, the canonical dual problem P"^ is as follows: 

sup p\a) = -^i''{A + aB)-'i-fia ^^^^ 
subject to cr € J" = {cr > I ^ + (jP ^ 0}, 

assuming that J- is non-empty. To ensure that {A + aB) is invertible, we have assumed that 
A and B are symmetric and non-zero. Again, the positive definiteness of {A + aB) for the 
dual problem has been found to be a sufficient condition, and it is not known yet whether it 
is a necessary condition. 

The following theorem appeared in |25j: 



^Thanks to Shu-Cherng Fang (NCSU, USA) for his input here. 
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Theorem 22. (Strong duality theorem) When the maximum value P {a*) in is finite, 
strong duality between the primal problem Pq and the dual problem P'^ holds, and the optimal 
solution for Pq is given by X* = {A + a*B)-H. 

However, the pair {Pq, P'^) may not be tight duals (i.e., they may not belong to the class 
TD). To show that {Pq, P'^) E TD, it also needs to be shown that the dual of P'^ is Pq. 
(Recall Remark [101) 

Remark 23. We have not provided a detailed analysis of how the canonical problem can be 
derived. The interested reader is referred to Our goal is to illustrate an instance of a 

pair of primal and dual problems that obey strong duality, and where an optimal solution can 
be obtained efficiently (in polynomial time). 

Strong duality for a variation of QCQP was established by More and Sorensen in 1983 [IB] 
(as described in [1]); they call theirs as the Trust Region Problem (TRP). The problem is to 
minimize a quadratic function qq, subject to a norm constraint x"^x < 5^ . This is an example 
of a non-convex problem where strong duality holds. Rendl and Wolkowicz [23] show that 
the TRP can be reformulated as an unconstrained concave maximization problem, and hence 
can be solved in polynomial time by using the interior point method |20j . 

Recall that decision problems are those with Yes/No answers. Combining Theorem 1221 with 
Lemma [9] above, we can conclude that 

Remark 24. Decision versions of Problems Pq in Iil0\) and in ill]) are members of the 
intersection class NP CoNP. 

Furthermore, from Theorem 3 in [25j, it is easy to see that problems Pq and P'^ can be solved 
in polynomial time. The authors in [25] use what they call a "boundarification" technique 
which moves the analytical solution x to a global minimizer x* on the boundary of the primal 
domain. This strengthens the conjecture that 

Conjecture 25. If two optimization problems Pa and Pf, are such that one of them is the 
dual of the other, that is, they exhibit strong duality, then both are polynomially solvable. 

As hinted earlier, the canonical dual feasible space in (jlip may be empty. In such a 
case, strong duality still holds on a feasible domain in which the matrix {A + aB) is not 
definite. However, the question as to how to solve the canonical dual problem is still open. 
It is conjectured in [1.2] that the primal problem (from which the canonical dual problem is 
derived) could be NP-hard. 

The observations above are for quadratic programming problems with a single quadratic 
constraint. It would be interesting to see what happens for quadratic programming problems 
with two constraints, whether strong duality still holds, and whether both the primal and the 
dual are still polynomially solvable. 

Semidefinite Programming (SDP). Ramana |22J exhibited strong duality for the SDP problem. 
However, the complexity of SDP is unknown; it was shown in |22j that the decision version 
of SDP is NP-complete if and only if NP = CoNP. 

( Note : There have been some published papers which say that SDP is polynomial time 
solvable. This is NOT correct, as the above result in [22] shows.) 
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6 Descriptive Complexity and Fixed Points 



On a final note, we would like to briefly describe a similar phenomenon which occurs in the fleld 
of Descriptive Complexity, which is the application of Finite Model theory to computational 
complexity. In particular, we would like to mention least fixed point (LFP) computation. A 
full description would be beyond the scope of this paper. However, we would like to briefiy 
mention a few related concepts and phenomena. 

For a good description of least fixed points (LFP) in existential second order (ESO) logic, the 
reader is referred to [14j (chapters 2 and 3) and |8j. If the input structures are ordered, then 
expressions in LFP logic can describe polynomial time (PTIME) computation |14j . 

The input instance to an LFP computation consists of a structure A, which includes a domain 
set A and a set of (first order) relations Ri, each with arity rj, 1 < i < J. The LFP 
computation works by a stagewise addition of tuples from A, to a new relation P (of some arity 
k). If Pi represents the relation (set of tuples) after stage i, then Pi C The transition 

from Pi to Pi+i is through an operator $, such that Pj+i = ^{Pi). At the beginning, P is 
empty, that is, Pq = 0. For some value of i, say when i = f , if Pj = Pf+i, a fixed point has 
been reached. 

Without going into details, let us just say that such a fixed point, reached as above, is also 
a least fixed point (LFP) if the operator $ can be chosen in a particular manner. The 
interested reader is referred to [2] (chapter 2) for details. 

Note that the number of elements in P can be at most \A\^ (where \A\ is the number of 
elements in A), which is polynomial in the size of the domain. Hence / < \A\^ , so an LFP is 
achieved within a polynomial number of stages. 

Similar to LFP, we can also define a greatest fixed point (GFP). This is obtained by doing 
the reverse; we start with the entire set A^ of /c-ary tuples from the universe A, and then 
removing tuples from P in stages. At the beginning, Pq = A^ . In further stages, Pi D Pi+i- 
The GFP is reached at stage g ii Pg = Pg+i- 

The logic that includes LFP and GFP expressions is known as LFP logic. It expresses 
decision problems (those with a Yes/No answer), such as those in Def. [T] and HI To be feasible, 
a solution should also obey the objective function constraint (/(x) > i^T or /(x) < K). 

The LFP computation expresses decision problems based on maximization. Before the fixed 
point is reached, the solution is infeasible; that is, the number of tuples in the fixed point 
relation P is insufficient. However, once the fixed point is reached, the solution becomes 
feasible. Similarly, the GFP computation expresses decision problems based on minimization. 

Problem. An interesting problem arising in LFP Logic is this: For what type of primal-dual 
optimization problem pairs will the LFP and GFP computation meet at the same fixed point? 
Does this mean that such a pair is polynomially solvable? 

7 Conclusion and Further Study 

Let us again stress that this is not a survey on Lagrangian duality or any other form of 
optimization duality. Rather, this is a survey on the connections and relationships between 
the computational complexity of optimization problems and duality. 
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In this paper, we have touched the tip of the iceberg on a very interesting problem, that of 
connecting the computational hardness of an optimization problem with its duality charac- 
teristics. A lot more study is required in this area. 

Another issue is that of saddle point for Lagrangian duals. This is a decidable problem; we 
can do brute force and find the primal and dual optimal solutions; this will tell us if there is 
a duality gap. If the gap is zero, then there is a saddle point. 

(Jeroslow [16] showed that the integer programming problem with quadratic constraints is 
undecidable if the number of variables is unbounded, which is an extreme condition. However, 
if each variable has a finite upper and lower bound, then the number of solutions is finite and 
thus it is possible to determine the best solution in finite time.) 

However, this problem would be NP-complete, unless we can tell whether it has a saddle point 
by looking at the structure of the problem or by running a polynomial time algorithm. 

We hope that this paper will motivate further research in this very interesting topic. 
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